Papers with answering questions
DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text (2024.findings-naacl)
Copied to clipboard
| Challenge: | Retrievalaugmented LLMs have been used to ground LLM in external knowledge . a gap exists in the current landscape regarding the effectiveness of grounding LLM on heterogeneous knowledge sources. |
| Approach: | They propose a model that uses symbolic language to generate symbolic queries . they use a dataset that is generated using predefined reasoning chains and human annotation . |
| Outcome: | The proposed model outperforms previous approaches by a significant margin in QA tasks over text. |
Table Question Answering for Low-resourced Indic Languages (2024.emnlp-main)
Copied to clipboard
| Challenge: | TableQA is the task of answering questions over tables of structured information, returning individual cells or tables as output. |
| Approach: | They propose a fully automatic large-scale tableQA data generation process for low-resource languages with limited budget. |
| Outcome: | The proposed method outperforms state-of-the-art LLMs on two Indic languages with no tableQA datasets and models on different aspects including mathematical reasoning capabilities and zero-shot cross-lingual transfer. |
AI for Science in the Era of Large Language Models (2024.emnlp-tutorials)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have demonstrated significant prowess in tasks involving natural language, such as translating languages, constructing chatbots, and answering questions. |
| Approach: | This tutorial explores the application of large language models to three crucial categories of scientific data: 1) textual data, 2) biomedical sequences, and 3) brain signals. |
| Outcome: | This tutorial will explore the application of large language models to three crucial categories of scientific data. |
Open-Domain Question Answering (2020.acl-tutorials)
Copied to clipboard
| Challenge: | tutorial provides a comprehensive overview of cutting-edge research in open-domain question answering (QA) |
| Approach: | tutorial provides a comprehensive overview of cutting-edge research in open-domain question answering . focus will shift to cutting- edge models proposed for open- domain QA . |
| Outcome: | The tutorial will cover cutting-edge research in open-domain question answering (QA) it will cover two-stage retriever-reader approaches, dense retriever and end-to-end training, and retriever free methods . |
Book QA: Stories of Challenges and Opportunities (D19-58)
Copied to clipboard
| Challenge: | Existing approaches to answer questions based on the full text of books are limited by their unique characteristics. |
| Approach: | They propose a system for answering questions based on the full text of books . they use a memory network to reason and predict an answer, and a novel question generator to improve generalization. |
| Outcome: | The proposed system improves on the recently published NarrativeQA corpus on Who questions . it shows that the proposed system is highly challenging and needs more research . |
Reinforced Extractive Summarization with Question-Focused Rewards (P18-3)
Copied to clipboard
| Challenge: | Existing methods for extractive summarization use human abstracts to create annotations for extraction units. |
| Approach: | They propose a method where abstracts are converted to Cloze-style comprehension questions to generate extractive summarization. |
| Outcome: | The proposed method surpasses state-of-the-art systems on the standard summarization dataset. |
‘Am I the Bad One’? Predicting the Moral Judgement of the Crowd Using Pre–trained Language Models (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing studies on NLP touch upon moral contexts in text. |
| Approach: | They construct a dataset that can be used for moral judgement tasks on a popular reddit subreddit. |
| Outcome: | The proposed model passes moral judgements on posts from a popular reddit subreddit . it shows that the model can be fine tuned and improves across the datasets . |
QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing question answering systems lack the ability to access relevant knowledge and reason over it. |
| Approach: | They propose a model that uses KGs to identify relevant knowledge in QA contexts and perform joint reasoning over them. |
| Outcome: | The proposed model improves on the CommonsenseQA and OpenBookQA datasets and performs interpretable and structured reasoning. |
Triggering Multi-Hop Reasoning for Question Answering in Language Models using Soft Prompts and Random Walks (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods that decompose multi-hop questions into single hop sub-questions are difficult to implement. |
| Approach: | They propose to use random-walks to guide pre-trained language models to map multi-hop questions to random-walked paths that lead to the answer. |
| Outcome: | The proposed methods improve on two T5 LMs. |
Using Commonsense Knowledge to Answer Why-Questions (2022.emnlp-main)
Copied to clipboard
Yash Kumar Lal, Niket Tandon, Tanvi Aggarwal, Horace Liu, Nathanael Chambers, Raymond Mooney, Niranjan Balasubramanian
| Challenge: | Existing approaches to integrating commonsense knowledge into large language models are implicit and explicit. |
| Approach: | They analyze the effects of model size and methods of injecting knowledge into TellMeWhy datasets to determine what aspects of commonsense knowledge are available in large language models. |
| Outcome: | The largest models yield substantial improvements over base models, but the amount of improvement decreases with larger model size. |
Probing the Multi-turn Planning Capabilities of LLMs via 20 Question Games (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are effective at answering clear questions but when faced with ambiguous queries they act unpredictably and produce incorrect outputs. |
| Approach: | They propose to use a surrogate problem to assess an LLMs’s ability to deduce an entity unknown to itself, but revealed to a judge, by asking the judge a series of queries. |
| Outcome: | The proposed model outperforms human players on the entity-deducing task by a large margin. |
EchoSight: Advancing Visual-Language Models with Wiki Knowledge (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge-based visual question answering systems struggle with these tasks due to limited integration of external knowledge. |
| Approach: | They propose a framework that enables large language models to answer visual questions requiring encyclopedic knowledge. |
| Outcome: | The proposed framework improves retrieval outcomes and accuracy of knowledge-based visual question answering tasks. |
M3: A Multi-View Fusion and Multi-Decoding Network for Multi-Document Reading Comprehension (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for multi-document reading comprehension cannot make full of the advantages of both approaches. |
| Approach: | They propose a multi-view fusion and multi-decoding method that integrates multiple documents for answering questions. |
| Outcome: | The proposed method improves on two mainstream multi-document reading comprehension datasets. |
Capturing Row and Column Semantics in Transformer Based Question Answering over Tables (2021.naacl-main)
Copied to clipboard
Michael Glass, Mustafa Canim, Alfio Gliozzo, Saneem Chemmengath, Vishwajeet Kumar, Rishav Chakravarti, Avi Sil, Feifei Pan, Samarth Bharadwaj, Nicolas Rodolfo Fauceglia
| Challenge: | Existing transformer based approaches have been used to answer questions over tables. |
| Approach: | They propose a transformer based architecture that independently classifies rows and columns to identify relevant cells and a model that incorporates existing tables to improve efficiency. |
| Outcome: | The proposed model outperforms the state-of-the-art transformer based approaches on WikiSQL lookup questions and achieves 3.4% and 18.86% additional precision improvement on the standard WikisQL benchmark. |
Triad: A Framework Leveraging a Multi-Role LLM-based Agent to Solve Knowledge Base Question Answering (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances with LLMs have shown promising results across various tasks, but their use in answering questions from knowledge bases remains largely unexplored. |
| Approach: | They propose a framework that utilizes an LLM-based agent with multiple roles for KBQA tasks. |
| Outcome: | The proposed framework outperforms state-of-the-art systems on the LC-QuAD and YAGO-QA benchmarks yielding F1 scores of 11.8% and 20.7%, respectively. |
GCRC: A New Challenging MRC Dataset from Gaokao Chinese for Explainable Evaluation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing machine reading comprehension datasets lack an explainable evaluation of systems' reasoning capabilities. |
| Approach: | They propose a dataset with multi-choice questions that evaluates MRC systems' reasoning process . they use sentence-level relevant supporting facts, error reason of distractors to evaluate MRC . |
| Outcome: | The proposed dataset is more challenging and useful for identifying limitations of existing MRC systems in an explainable way. |
Are LLMs Aware that Some Questions are not Open-ended? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown impressive capability to answer questions in a wide range of scenarios. |
| Approach: | They propose a method that enhances the question awareness of LLMs by adaptively adjusting the output distributions based on question features. |
| Outcome: | The proposed method improves the question awareness of LLMs by adaptively adjusting the output distributions based on question features. |
CORN: Co-Reasoning Network for Commonsense Question Answering (2022.coling-1)
Copied to clipboard
| Challenge: | Existing work uses two independent modules to model QA content and external commonsense knowledge graph (KG) Existing research uses two separate modules to create QA contextual text representations and relationships between QA entities. |
| Approach: | They propose a commonsense question answering (QA) model that uses two independent modules to model QA contextual text representation and relationships between QA entities in KG. |
| Outcome: | The proposed model achieves state-of-the-art on QA benchmarks in the CommonsenseQA and OpenBookQA datasets. |
QA Domain Adaptation using Hidden Space Augmentation and Self-Supervised Contrastive Adaptation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Question answering models often suffer from performance deterioration upon deployment . |
| Approach: | They propose a self-supervised framework called QADA for QA domain adaptation . they propose to augment training QA samples with hidden space augmentation . |
| Outcome: | The proposed framework improves on multiple target datasets over state-of-the-art methods. |
Domain Adaptation for Question Answering via Question Classification (2022.coling-1)
Copied to clipboard
| Challenge: | Question answering systems often experience performance deterioration upon user-generated questions. |
| Approach: | They propose a question classification framework to help QA domains adapt to different domains. |
| Outcome: | The proposed framework improves on state-of-the-art datasets against multiple datasets. |
Think Visually: Question Answering through Virtual Imagery (P18-1)
Copied to clipboard
| Challenge: | Existing models of geometric reasoning are based on visual representations of objects and objects, but they are not based in symbols or words. |
| Approach: | They propose a new deep network architecture that specializes in answering questions that admit latent visual representations and learns to generate and reason over such representations. |
| Outcome: | The proposed model can generate and reason over latent visual representations and is validated by two synthetic benchmarks. |
TABBIE: Pretrained Representations of Tabular Data (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing work on tabular representation-learning jointly models tables and associated text using self-supervised objective functions derived from pretrained language models such as BERT. |
| Approach: | They propose a tabular representation-learning model that integrates tabular data with a pretraining objective function that detects corrupted cells. |
| Outcome: | The proposed model understands complex table semantics and numerical trends. |
ChatGPT Is a Knowledgeable but Inexperienced Solver: An Investigation of Commonsense Problem in Large Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | acquiring and representing commonsense in machines has posed a long-standing challenge (Li et al., 2021; Zhang e t al, 2022; Zhou e al. 2023) . |
| Approach: | They use a commonsense-based LLM to evaluate ChatGPT's commonsensing abilities by analyzing 11 datasets and generating knowledge descriptions. |
| Outcome: | The proposed model can achieve good QA accuracies while still struggling with certain domains of datasets. |
RADAR: A Reasoning-Guided Attribution Framework for Explainable Visual Data Analysis (2026.findings-eacl)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) provide no visibility into which parts of visual data informed their conclusions. |
| Approach: | They propose a semi-automatic approach to attribute reasoning process by highlighting regions in charts and graphs that justify model answers. |
| Outcome: | The proposed method improves attribution accuracy by up to 15 percentage points compared to baseline methods and achieves high semantic similarity with ground truth responses. |
TabSQLify: Enhancing Reasoning Capabilities of LLMs Through Table Decomposition (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models struggle with large tables due to their limited input length . a novel method that decomposes tables into smaller and relevant sub-tables reduces the computational load on LLMs . |
| Approach: | They propose a method that leverages text-to-SQL generation to decompose tables into smaller and relevant sub-tables . the method can reduce the input context length significantly, making it more scalable and efficient . |
| Outcome: | The proposed method performs remarkably well on the WikiTQ benchmark and on the TabFact benchmark. |
MultiTabQA: Generating Tabular Answers for Multi-Table Question Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Recent tabular question answering models only answer questions over a single table . multi-table operations often result in tabular outputs . |
| Approach: | They propose a model that answers questions over multiple tables and generalizes to generate tabular answers. |
| Outcome: | The proposed model outperforms state-of-the-art single table QA models on a multi-table QA setting. |
Dense-Caption Matching and Frame-Selection Gating for Temporal Localization in VideoQA (2020.acl-main)
Copied to clipboard
| Challenge: | Recent years have witnessed a paradigm shift in the way we get our information, and a lot of it. |
| Approach: | They propose a video question answering model which integrates multi-modal input sources and finds temporally relevant information to answer questions. |
| Outcome: | The proposed model outperforms the state-of-the-art on a TVQA dataset. |
Query Refinement Prompts for Closed-Book Long-Form QA (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) can answer questions and produce long-form texts, but the latter is difficult to evaluate since they are subjective in nature. |
| Approach: | They propose query refinement prompts that encourage LLMs to express multifacetedness and generate long-form answers covering multiple facets of the question. |
| Outcome: | The proposed model outperforms fully finetuned models in the closed-book setting and retrieve-then-generate open-book models. |
MAPWise: Evaluating Vision-Language Models for Advanced Map Queries (2025.naacl-long)
Copied to clipboard
| Challenge: | Vision-language models excel at tasks requiring joint understanding of visual information and natural language. |
| Approach: | They propose to use choropleth maps to answer questions from three geographical regions in the United States, India, China as question templates. |
| Outcome: | The proposed model outperforms other models in the area of visual language and visual question answering. |
TRACE the Evidence: Constructing Knowledge-Grounded Reasoning Chains for Retrieval-Augmented Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing retrievers are not perfect and often include irrelevant documents in the retrieved set. |
| Approach: | They propose to construct knowledge-grounded reasoning chains from retrieved documents to integrate supporting evidence into RAG models. |
| Outcome: | The proposed model achieves an average performance improvement of 14.03% on three multi-hop QA datasets. |
Think about it! Improving defeasible reasoning by first modeling the question scenario. (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature suggests that a person forms a mental model of the problem scenario before answering questions. |
| Approach: | They propose to have a model first create a graph of relevant influences and leverage that graph as an additional input when answering a defeasible query. |
| Outcome: | The proposed model achieves state-of-the-art on three different defeasible reasoning datasets. |
Fine-grained Knowledge Enhancement for Retrieval-Augmented Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies rely on semantic similarity to retrieve knowledge but ignore fine-grained information within documents. |
| Approach: | They propose a fine-grained knowledge enhancement method to fill knowledge gaps with retrieved external information by a Chain-of-Thought prompting procedure and a decoding enhancement strategy to constrain the document-based decoding process. |
| Outcome: | The proposed method can be applied in a plug-and-play manner to enhance its performance with no additional modules or training process. |
Towards Robust Numerical Question Answering: Diagnosing Numerical Capabilities of NLP Systems (2022.emnlp-main)
Copied to clipboard
| Challenge: | Numerical Question Answering is the task of answering questions that require numerical capabilities. |
| Approach: | They propose to conduct numerical capability diagnosis on a series of Numerical Question Answering systems and datasets. |
| Outcome: | The proposed approach relieves existing systems’ lack of robust numerical capabilities. |
Fine-tuning Smaller Language Models for Question Answering over Financial Documents (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent research has shown that smaller language models can acquire substantial reasoning abilities when fine-tuned with reasoning exemplars crafted by a significantly larger teacher model. |
| Approach: | They propose to fine-tune several smaller model to generate programs that encode the required financial reasoning and calculations. |
| Outcome: | The proposed model outperforms the teacher model in the financial domain by adjusting the entity extraction for the specific data format. |
Data-QuestEval: A Referenceless Metric for Data-to-Text Semantic Evaluation (2021.emnlp-main)
Copied to clipboard
Clement Rebuffel, Thomas Scialom, Laure Soulier, Benjamin Piwowarski, Sylvain Lamprier, Jacopo Staiano, Geoffrey Scoutheeten, Patrick Gallinari
| Challenge: | QuestEval is a metric used in text-to-text tasks, but its adaptation to Data-to Text tasks requires multimodal Question Generation and Answering systems, which are seldom available. |
| Approach: | They propose to build synthetic multimodal corpora enabling to train multimodal components for a data-QuestEval metric. |
| Outcome: | The proposed method obtains state-of-the-art correlations with human judgment on the WebNLG and WikiBio benchmarks. |
MuKA: Multimodal Knowledge Augmented Visual Information-Seeking (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods for visual information-seeking tasks rely on textual knowledge . existing methods can impair information retrieval and confuse MLLMs . |
| Approach: | They propose a framework which leverages a multimodal knowledge base to address these limitations. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on the InfoSeek and E-VQA benchmarks. |
CAFE: Retrieval Head-based Coarse-to-Fine Information Seeking to Enhance Multi-Document QA Capability (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to extend context length of Large Language Models (LLMs) still struggle with retrieval and reasoning in long context inputs. |
| Approach: | They propose a coarse-to-fine method to enhance multi-document question-answering capacities by removing background and distracting documents. |
| Outcome: | Experiments show that CAFE outperforms baseline methods on multiple documents. |
CRIPP-VQA: Counterfactual Reasoning about Implicit Physical Properties via Video Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | Videos often capture objects, their visible properties, their motion, and the interactions between different objects. |
| Approach: | They propose a video question answering dataset for reasoning about the implicit physical properties of objects in a scene. |
| Outcome: | The proposed dataset enables evaluation under several out-of-distribution settings – videos with objects with masses, coefficients of friction, and initial velocities that are not observed in the training distribution. |
Ontology-Guided Reverse Thinking Makes Large Language Models Stronger on Knowledge Graph Question Answering (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods rely on entity vector matching, but the purpose of the question is abstract and difficult to match with specific entities. Existing approaches rely only on entity-vector matching, and there is a problem with multi-hop reasoning. |
| Approach: | They propose a framework that constructs reasoning paths from purposes back to conditions using the KG ontology. |
| Outcome: | Experiments on the WebQSP and CWQ datasets show that ORT significantly improves the capability of large language models in knowledge graph question answering tasks (KGQA). |
What is the Real Intention behind this Question? Dataset Collection and Intention Classification (2023.acl-long)
Copied to clipboard
| Challenge: | Using the Wikipedia discussions, we identified positive/neutral and negative intentions in questions . questions can also reflect implicit offenses such as highlighting one’s lack of knowledge or bolstering an alleged superior knowledge, which can lead to conflict in conversations. |
| Approach: | They propose to use a dataset to identify questions with positive/neutral and negative intentions and the underlying intention categories within each group to highlight tacit and apparent intents. |
| Outcome: | The proposed method highlights tacit and apparent intents and uses Transformers augmented by TF-IDF-based features to classify the main intention categories. |
AnchorCoT: Anchors Pave the Way for Multi-hop Reasoning (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated potential reasoning capabilities through prompt design, such as the Chain of Thought (CoT). |
| Approach: | They propose a new reasoning approach that predicts key entities which work as important “anchors” and employs a ranking algorithm to ensure the logical sequence of the predicted answers. |
| Outcome: | The proposed approach outperforms existing methods in multi-hop question reasoning and provides more accurate reasoning results in multihop question answering tasks. |
Language Guided Visual Question Answering: Elevate Your Multimodal Language Model Using Knowledge-Enriched Prompts (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Visual question answering (VQA) is a task that requires an understanding of both the image and the question to provide a natural language answer. |
| Approach: | They propose a multimodal framework that leverages language guidance to answer questions more accurately. |
| Outcome: | The proposed framework improves on the multi-choice question-answering task using CLIP and BLIP models. |
Measuring Retrieval Complexity in Question Answering Systems (2024.findings-acl)
Copied to clipboard
| Challenge: | a new metric, retrieval complexity (RC), measures the difficulty of answering questions. |
| Approach: | They propose a retrieval complexity metric conditioned on the completeness of retrieved documents . they propose an unsupervised pipeline to measure RC given an arbitrary retrieval system . |
| Outcome: | The proposed pipeline measures RC more accurately than alternative estimators on six challenging QA benchmarks. |
Reflection on Knowledge Graph for Large Language Models Reasoning (2025.findings-acl)
Copied to clipboard
Yigeng Zhou, Wu Li, Yifan Lu, Jing Li, Fangming Liu, Meishan Zhang, Yequan Wang, Daojing He, Honghai Liu, Min Zhang
| Challenge: | Existing methods for supplementing Large Language Models (LLMs) with knowledge graphs often introduce noise in the retrieval and reasoning pipeline, hindering their ability to integrate external knowledge for complex multi-hop question answering. |
| Approach: | They propose a framework to enhance LLMs' reasoning capabilities through reflective engagement with knowledge graphs by Query Decoupling, LLM-Driven Knowledge Graph Exploration, and Inference with Knowledge Reconstruction. |
| Outcome: | The proposed framework integrates external knowledge into LLMs and trains them to leverage this knowledge for answering questions. |
EduAdapt: A Question Answer Benchmark Dataset for Evaluating Grade-Level Adaptability in LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing models produce outputs that are too advanced or vague for younger learners and there are no standardized benchmarks to evaluate their ability to adapt across cognitive and developmental stages. |
| Approach: | They propose to use a benchmark to assess LLMs' ability to adapt to different grade levels and to use it to evaluate their model's performance. |
| Outcome: | The proposed framework assesses the ability of large language models to adapt to grade levels across a range of subjects and grades. |
FoREST: Frame of Reference Evaluation in Spatial Reasoning Tasks (2025.emnlp-main)
Copied to clipboard
| Challenge: | Spatial reasoning is a fundamental aspect of human intelligence. |
| Approach: | They propose a framework to assess FoR comprehension in large language models (LLMs) by using the Frame of Reference Evaluation in Spatial Reasoning Tasks benchmark. |
| Outcome: | The proposed method improves overall performance across spatial reasoning tasks. |
MegaRAG: Multimodal Knowledge Graph-Based Retrieval Augmented Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing RAG solutions for large language models are limited by context windows limiting their ability to process long-form, domain-specific content. |
| Approach: | They propose a multimodal knowledge graph-based RAG that enables cross-modal reasoning . their method incorporates visual cues into the construction of knowledge graphs, retrieval phase, and answer generation process . |
| Outcome: | Experimental results show that the proposed approach outperforms existing approaches on textual and multimodal benchmarks. |